ENCODER-DECODER ARCHITECTURE

Exploring Interpretability in Event Extraction: Multitask Learning of a Neural Event Classifier and an Explanation Decoder (2020.acl-srw)

Challenge:	Existing factoid question answering systems rely on annotated datasets such as SimpleQuestions to generate questions from knowledge graphs.
Approach:	They propose a neural model that generates questions from knowledge graphs triples in a “zero-shot” setup.
Outcome:	The proposed model outperforms state-of-the-art on this task.

Copied to clipboard

Zheng Tang, Gus Hahn-Powell, Mihai Surdeanu

Information Extraction & Text Mining

Challenge:	EE is a key requirement for machine learning in many domains, e.g., legal, medical, finance.
Approach:	They propose an interpretable approach for event extraction that jointly trains a classifier and a rule decoder for event processing.
Outcome:	The proposed approach can be used for semi-supervised learning and its performance improves when trained on automatically-labeled data generated by a rule-based system.

Multilingual, Multi-scale and Multi-layer Visualization of Intermediate Representations (D19-3)

Copied to clipboard

Carlos Escolano, Marta R. Costa-jussà, Elora Lacroux, Pere-Pau Vázquez

Data-to-text Generation with Macro Planning (2021.tacl-1)

Challenge:	Currently, the main alternatives to deal with sequences are Recurrent Neural Networks (RNN) architectures and the Transformer.
Approach:	They propose a web-based tool that visualizes the sentence and token representations of RNNs and Transformer architectures at the sentence level.
Outcome:	The proposed visualization tool analyses gender inequalities in contextual word embeddings and the common language representation in a multilingual machine translation system.

Copied to clipboard

Ratish Puduppully, Mirella Lapata

Text Generation

Challenge:	Recent approaches to data-to-text generation adopt the encoder-decoder architecture . however, these models perform poorly at selecting appropriate content and ordering it coherently .
Approach:	They propose a neural model with a macro planning stage followed by a generation stage . they use data from databases of records, simulations of physical systems, accounting spreadsheets .
Outcome:	The proposed model outperforms baselines on two data-to-text benchmarks . it uses the encoderdecoder architecture and is compared with existing models .

Generating Equation by Utilizing Operators : GEO model (2020.coling-main)

Copied to clipboard

Kyung Seo Ki, Donggeon Lee, Bugeun Kim, Gahgene Gweon

Is Encoder-Decoder Redundant for Neural Machine Translation? (2022.aacl-main)

Challenge:	Existing neural models that use hand-crafted features are expensive and lack domain-specific knowledge.
Approach:	They propose a GEO model that uses operator-based features to generate equations using natural language sentences.
Outcome:	The proposed model outperforms state-of-the-art models on two datasets and 82.1% in ALG514.

Copied to clipboard

Yingbo Gao, Christian Herold, Zijian Yang, Hermann Ney

LOCOST: State-Space Models for Long Document Abstractive Summarization (2024.eacl-long)

Challenge:	Encoder-decoder architecture is widely adopted for sequence-to-sequence modeling tasks.
Approach:	They propose to combine bilingual and multilingual translations to train a language model to do translation.
Outcome:	The proposed approach performs on par with the baseline encoder-decoder Transformer . the proposed approach is compared with the translation model in the target language .

Copied to clipboard

Florian Le Bronnec, Song Duong, Mathieu Ravaut, Alexandre Allauzen, Nancy Chen, Vincent Guigue, Alberto Lumbreras, Laure Soulier, Patrick Gallinari

Information Extraction & Text Mining

Challenge:	State-space models are a low-complexity alternative to transformers for text generation . however, the quadratic complexity of the input length restricts the application of large pretrained models to long texts.
Approach:	They propose an encoder-decoder architecture based on state-space models for conditional text generation with long context inputs.
Outcome:	The proposed model saves memory and memory during training and inference time while saving 50% and 87% of memory.

KAT: A Knowledge Augmented Transformer for Vision-and-Language (2022.naacl-main)

Copied to clipboard

Liangke Gui, Borui Wang, Qiuyuan Huang, Alexander Hauptmann, Yonatan Bisk, Jianfeng Gao

CiteBART: Learning to Generate Citations for Local Citation Recommendation (2025.emnlp-main)

Challenge:	Existing methods for knowledge retrieval and answer prediction have left open questions about the quality and relevance of the retrieved knowledge and how the reasoning processes over implicit and explicit knowledge should be integrated.
Approach:	They propose a Knowledge Augmented Transformer which integrates both implicit and explicit knowledge in an encoder-decoder architecture while simultaneously reasoning over both knowledge sources during answer generation.
Outcome:	The proposed model achieves a strong state-of-the-art (+6% absolute) on the open-domain multimodal task of OK-VQA.

Copied to clipboard

Ege Yiğit Çelik, Selma Tekir

SEAM: Bridging the Temporal-Semantic Granularity Gap for LLM-based Speech Recognition (2026.findings-eacl)

Challenge:	Local citation recommendation (LCR) suggests a set of papers for a citation placeholder in a given context.
Approach:	They introduce a citation-specific pre-training framework where author-date citation tokens are masked to learn to reconstruct them to fulfill local citation recommendation (LCR).
Outcome:	The proposed model achieves state-of-the-art performance on LCR benchmarks except for the FullTextPeerRead dataset, which is quite small to see the advantage of generative pre-training.

Copied to clipboard

Junseok Oh, Ji-Hwan Kim

Disentangling Language and Knowledge in Task-Oriented Dialogs (N19-1)

Challenge:	Existing duration-based methods generate embeddings at fixed rates, creating distributional mismatch with LLM pre-training.
Approach:	They propose an encoder-decoder architecture that generates embeddings at variable rates through cross-attention between speech features and text embeddables.
Outcome:	The proposed architecture achieves competitive performance on LibriSpeech (2.6%/5.2% WER) and 4.7% WER on TED-LIUM-v2 with a multi-stage training strategy and First Token Guidance.

Copied to clipboard

Dinesh Raghu, Nikhil Gupta, null Mausam

Dialogue Systems & Conversational Agents

Challenge:	Existing approaches to handle task-oriented dialogs break when asked to handle such changes.
Approach:	They propose an encoder-decoder architecture with a novel Bag-of-Sequences memory which facilitates the disentangled learning of the response’s language model and its knowledge incorporation.
Outcome:	The proposed architecture outperforms state-of-the-art models on bAbI OOV test sets and other human-human datasets and shows that it is robust to KB modifications.

Context Sensitive Neural Lemmatization with Lematus (N18-1)

Copied to clipboard

Toms Bergmanis, Sharon Goldwater

Challenge:	a new contextsensitive lemmatizer is designed to improve performance on unseen and ambiguous words.
Approach:	They propose a context-sensitive lemmatizer which incorporates character-level sentence context.
Outcome:	The proposed model outperforms the best competitor in a data setting and lower-resource setting . the model includes context for ambiguous words, but the latter has a greater effect on performance .

PV2TEA: Patching Visual Modality to Textual-Established Information Extraction (2023.findings-acl)

Copied to clipboard

Hejie Cui, Rongmei Lin, Nasser Zalmout, Chenwei Zhang, Jingbo Shang, Carl Yang, Xian Li

Bootstrapping Generators from Noisy Data (N18-1)

Challenge:	Empirical results show up to 11.74% absolute (20.97% relative) increase over unimodal baselines.
Approach:	They propose to patch the visual modality to the textual-established attribute in- formation extractor.
Outcome:	Empirical results show up to 11.74% absolute (29.9% relative) increase over unimodal baselines.

Copied to clipboard

Laura Perez-Beltrachini, Mirella Lapata

Deep Communicating Agents for Abstractive Summarization (N18-1)

Challenge:	Existing methods for data-to-text generation focus on learning correspondences between structured data and associated texts.
Approach:	They aim to bootstrap generators from large scale datasets where data and related texts are loosely aligned.
Outcome:	The proposed model improves on a vanilla encoder-decoder which relies on soft attention.

Copied to clipboard

Asli Celikyilmaz, Antoine Bosselut, Xiaodong He, Yejin Choi

Improving Human Text Comprehension through Semi-Markov CRF-based Neural Section Title Generation (N19-1)

Challenge:	Empirical results show that multiple communicating agents produce a better summary than extractive summarization.
Approach:	They propose an encoder-decoder architecture that uses deep communicating agents to represent a long document for abstractive summarization.
Outcome:	Empirical results show that multiple communicating agents produce a better summary than baselines.

Copied to clipboard

Sebastian Gehrmann, Steven Layne, Franck Dernoncourt

Challenge:	a study of long documents that do not include short sections in their titles shows that they improve comprehension and speed .
Approach:	They propose to extract the most salient sentence and apply deletion-based compression to generate section titles in low-resource environments.
Outcome:	The proposed approach outperforms other methods in low-resource environments while outperforming other approaches.

Expressing Visual Relationships via Language (P19-1)

Copied to clipboard

Hao Tan, Franck Dernoncourt, Zhe Lin, Trung Bui, Mohit Bansal

UReader: Universal OCR-free Visually-situated Language Understanding with Multimodal Large Language Model (2023.findings-emnlp)

Challenge:	Current studies on image captioning focus on single image, but there are no effective models for generating relational captions for two images.
Approach:	They propose a language-guided image editing dataset that contains real image pairs with corresponding editing instructions.
Outcome:	The proposed model outperforms baseline and existing methods on two datasets.

Copied to clipboard

Jiabo Ye, Anwen Hu, Haiyang Xu, Qinghao Ye, Ming Yan, Guohai Xu, Chenliang Li, Junfeng Tian, Qi Qian, Ji Zhang, Qin Jin, Liang He, Xin Lin, Fei Huang

Mapping Language to Code in Programmatic Context (D18-1)

Challenge:	Existing studies for visually-situated language understanding have shown shallow zero-shot visual text recognition ability when fed a low-resolution image with salient text information.
Approach:	They propose a model for universal OCR-free visually-situated language understanding based on the Multimodal Large Language Model (MLLM) their model is jointly finetuned on a wide range of visually situated language understanding tasks via a unified instruction format.
Outcome:	The proposed model achieves state-of-the-art ocr-free performance in 8 out of 10 visually-situated language understanding tasks across 5 domains: documents, tables, charts, natural images, and webpage screenshots.

Copied to clipboard

Srinivasan Iyer, Ioannis Konstas, Alvin Cheung, Luke Zettlemoyer

Programming Languages in NLP

Challenge:	Existing approaches for automatically mapping natural language to executable code have considered limited language or code environments.
Approach:	They propose a task of generating class member functions given English documentation and the programmatic context provided by the rest of the class.
Outcome:	The proposed model can generate member functions from documentation and the class environment.

Low-Resource Response Generation with Template Prior (D19-1)

Copied to clipboard

Ze Yang, Wei Wu, Jian Yang, Can Xu, Zhoujun Li

Text Generation

Challenge:	Existing open domain response generation models are limited to paired data, but are less explored in real-world applications.
Approach:	They propose to train a neural response generation model with unpaired data and paired data as prior.
Outcome:	The proposed model outperforms state-of-the-art models in both automatic and human evaluation when only a few pairs are available.

When and Where Did it Happen? An Encoder-Decoder Model to Identify Scenario Context (2024.findings-emnlp)

Copied to clipboard

Enrique Noriega-Atala, Robert Vacareanu, Salena Ashton, Adarsh Pyarelal, Clayton Morrison, Mihai Surdeanu

Learning Cooperative Interactions for Multi-Overlap Aspect Sentiment Triplet Extraction (2022.findings-emnlp)

Challenge:	a relatively small fine-tuned encoder-decoder model performs better than out-of-the-box LLMs and semantic role labeling parsers to accurately predict the relevant scenario information.
Approach:	They propose a neural architecture finetuned for the task of scenario context generation . they use a curated dataset of time and location annotations to train an encoder-decoder architecture .
Outcome:	The proposed model performs better than out-of-the-box LLMs and semantic role labeling parsers to accurately predict the relevant scenario information of a particular entity or event.

Copied to clipboard

Shiman Zhao, Wei Chen, Tengjiao Wang

Sentiment Analysis

Challenge:	Existing methods for aspect sentiment triplet extraction focus on the single interactions between an aspect and an opinion.
Approach:	They propose a multi-overlap triplet extraction method which decodes the complex relations between multiple aspects and opinions by learning their cooperative interactions.
Outcome:	The proposed method outperforms baselines, especially multi-overlap triplets.

ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking Inference (2022.findings-acl)

Copied to clipboard

Kai Hui, Honglei Zhuang, Tao Chen, Zhen Qin, Jing Lu, Dara Bahri, Ji Ma, Jai Gupta, Cicero Nogueira dos Santos, Yi Tay, Donald Metzler

Disentangling Text Representation With Counter-Template For Unsupervised Opinion Summarization (2023.findings-acl)

Challenge:	State-of-the-art neural models typically encode document-query pairs using cross-attention for re-ranking.
Approach:	They propose to fine tune a pretrained encoder-decoder model using document to query generation.
Outcome:	The proposed model achieves comparable results to more expensive approaches while being 6.8X faster.

Copied to clipboard

Yanyue Zhang, Deyu Zhou

Responsible & Trustworthy NLP

Challenge:	Existing approaches for unsupervised opinion summarization are based on reconstruction model, but selection is too coarse as not all information in each input is equally essential for the summary.
Approach:	They propose a framework for unsupervised opinion summarization based on text representation disentanglement with counter-template.
Outcome:	The proposed framework outperforms the state-of-the-art models on quality and stability on two benchmark datasets.

LearnerCoMPASS: Intelligent Tutoring System with Dynamic Cognitive Diagnosis and Multi-Model Path Planning (2026.acl-long)

Copied to clipboard

Ziji Sheng, Guiyao Tie, Weidong Wang, Pan Zhou, Daizong Liu

Learning Action-Effect Dynamics for Hypothetical Vision-Language Reasoning Task (2022.findings-emnlp)

Challenge:	Existing adaptive learning systems struggle to achieve deep personalization, dynamic adaptability, and content trustworthiness.
Approach:	They propose a framework that integrates large language models into adaptive learning systems . they propose 'cognitive multi-model planning adapted system' to enable deep personalization .
Outcome:	The proposed framework outperforms state-of-the-art learning paths and improves trustworthiness.

Copied to clipboard

Shailaja Keyur Sampat, Pratyay Banerjee, Yezhou Yang, Chitta Baral

Fine-Grained Controllable Text Generation Using Non-Residual Prompting (2022.acl-long)

Challenge:	‘Actions’ play a vital role in how humans interact with the world and therefore require the capability to perform ‘Reasoning about Actions & Change’ (RAC).
Approach:	They propose an encoder-decoder architecture to learn the representation of actions as vectors and combine it with existing modality parsers and a scene graph question answering model to evaluate the proposed system.
Outcome:	The proposed system performs better than baselines on the CLEVR_HYP dataset and compares with existing parsers and a scene graph question answering model.

Copied to clipboard

Fredrik Carlsson, Joey Öhman, Fangyu Liu, Severine Verlinden, Joakim Nivre, Magnus Sahlgren

Position Offset Label Prediction for Grammatical Error Correction (2022.coling-1)

Challenge:	Existing approaches to control the text generation process are not expressive enough.
Approach:	They propose an encoder-decoder architecture that enables intermediate text prompts at arbitrary time steps.
Outcome:	The proposed architecture is expressive and versatile on multiple experimental settings.

Copied to clipboard

Xiuyu Wu, Jingsong Yu, Xu Sun, Yunfang Wu

Syntactic Text Processing

Challenge:	Experimental results show that our proposed POL-Pc framework improves baseline models and yields consistent performance gain over various data augmentation methods.
Approach:	They propose a position offset label prediction subtask to integrate correction editing operations into a unified framework.
Outcome:	The proposed model outperforms baseline models on Chinese, English and Japanese datasets by a wide margin.

Structure-aware Fine-tuning of Sequence-to-sequence Transformers for Transition-based AMR Parsing (2021.emnlp-main)

Copied to clipboard

Jiawei Zhou, Tahira Naseem, Ramón Fernandez Astudillo, Young-Suk Lee, Radu Florian, Salim Roukos

PromptRank: Unsupervised Keyphrase Extraction Using Prompt (2023.acl-long)

Challenge:	Recent work shows that pre-trained sequence-to-sequence Transformer models are effective in predicting linearized Abstract Meaning Representation graphs.
Approach:	They propose a structure-aware transition-based approach to AMR parsing that integrates general pre-trained sequence-to-sequence language models with a structured transition set.
Outcome:	The proposed approach retains the desirable properties of previous approaches while reaching the new parsing state of the art for AMR 2.0.

Copied to clipboard

Aobo Kong, Shiwan Zhao, Hao Chen, Qicheng Li, Yong Qin, Ruiqi Sun, Xiaoyan Bai

TURNA: A Turkish Encoder-Decoder Language Model for Enhanced Understanding and Generation (2024.findings-acl)

Challenge:	Existing keyphrase extraction methods struggle with document and candidate length discrepancies or fail to fully utilize the pre-trained language model without further fine-tuning.
Approach:	They propose an unsupervised keyphrase extraction approach that uses a pre-trained language model to rank candidates based on document embeddings.
Outcome:	The proposed approach outperforms the existing keyphrase extraction approach on six benchmarks.

Copied to clipboard

Gökçe Uludoğan, Zeynep Balal, Furkan Akkurt, Meliksah Turker, Onur Gungor, Susan Üsküdarlı

Multitask Pretraining with Structured Knowledge for Text-to-SQL Generation (2023.acl-long)

Challenge:	Recent advances in natural language processing have favored well-resourced English-centric models, resulting in a significant gap with low-resource languages.
Approach:	They propose a language model for the low-resource language Turkish that is capable of both natural language understanding and generation tasks.
Outcome:	The proposed model outperforms multilingual models in understanding and generation tasks and competes with monolingual models for understanding tasks.

Copied to clipboard

Robert Giaquinto, Dejiao Zhang, Benjamin Kleiner, Yang Li, Ming Tan, Parminder Bhatia, Ramesh Nallapati, Xiaofei Ma

PASTE: A Tagging-Free Decoding Framework Using Pointer Networks for Aspect Sentiment Triplet Extraction (2021.emnlp-main)

Challenge:	Existing methods for learning representations of structured knowledge are limited to the minority of people with technical skills.
Approach:	They propose a large pretraining dataset and strategy for learning representations of text, tables, and SQL code that leverages the entire context of the problem.
Outcome:	The proposed model improves on two SQL tasks and shows a 1.7 and 2.2 percentage point improvement over existing methods.

Copied to clipboard

Rajdeep Mukherjee, Tapas Nayak, Yash Butala, Sourangshu Bhattacharya, Pawan Goyal

Syntactic Text Processing

Challenge:	Existing methods for tagging opinion triplets fail to capture the strong interdependence between the three opinion factors, whereas grid tabbing fails to capture span-level semantics while predicting sentiment between an aspect-opinion pair.
Approach:	They propose a tagging-free approach to extracting opinion triplets using a pointer network decoding framework that captures the interdependence between the three elements of an opinion triple.
Outcome:	The proposed architecture captures the interdependence between the aspect and opinion triplets while predicting their connecting sentiment.

BanglaTLit: A Benchmark Dataset for Back-Transliteration of Romanized Bangla (2024.findings-emnlp)

Copied to clipboard

Md Fahim, Fariha Shifat, Fabiha Haider, Deeparghya Barua, Md Sourove, Md Ishmam, Md Bhuiyan

XC-Cache: Cross-Attending to Cached Context for Efficient LLM Inference (2024.findings-emnlp)

Challenge:	low-resource languages like Bangla are limited by the lack of datasets.
Approach:	They propose a large-scale transliteration dataset and a pre-training corpus on romanized Bangla.
Outcome:	The proposed datasets show that the proposed methods can enrich romanized Bangla.

Copied to clipboard

Joao Monteiro, Étienne Marcotte, Pierre-Andre Noel, Valentina Zantedeschi, David Vazquez, Nicolas Chapados, Christopher Pal, Perouz Taslakian

Responsible & Trustworthy NLP

Challenge:	XC-Llama uses pre-trained decoder-only models to condition generation on reference text without the prompt.
Approach:	They propose a model that uses cross-attention to condition generation on reference text without the prompt.
Outcome:	The proposed models outperform prompt-based inference methods and reduce space footprint relative to standard KV caching by two orders of magnitude.

Controlled Transformation of Text-Attributed Graphs (2024.findings-emnlp)

Copied to clipboard

Nidhi Vakil, Hadi Amiri

Typos Correction Training against Misspellings from Text-to-Text Transformers (2024.lrec-main)

Challenge:	Graph generation is the process of generating new graphs with similar attributes to real world graphs.
Approach:	They propose a controllable multi-objective translation model for text-attributed graphs that can translate a given source graph to a target graph while satisfying multiple desired graph attributes at granular level.
Outcome:	The proposed model can translate a given source graph to a target graph while satisfying multiple desired graph attributes at granular level.

Copied to clipboard

Guicai Xie, Ke Zhang, Lei Duan, Wei Zhang, Zeqian Huang